Risk-sensitive semi-Markov decision problems with discounted cost and general utilities

Iterated risk measures for risk-sensitive Markov decision processes with discounted cost

We demonstrate a limitation of discounted expected utility, a standard approach for representing the preference to risk when future cost is discounted. Specifically, we provide an example of the preference of a decision maker that appears to be rational but cannot be represented with any discounted expected utility. A straightforward modification to discounted expected utility leads to inconsis...

Chapter 13 CONSTRAINED DISCOUNTED SEMI - MARKOV DECISION PROCESSES

Markov Decision Models with Weighted Discounted Criteria

We consider a discrete time Markov Decision Process with innnite horizon. The criterion to be maximized is the sum of a number of standard discounted rewards, each with a diierent discount factor. Situations in which such criteria arise include modeling investments, production, modeling projects of diierent durations and systems with multiple criteria, and some axiomatic formulations of multi-a...

Discounted Markov decision processes with utility constraints

-We consider utility-constrained Markov decision processes. The expected utility of the total discounted reward is maximized subject to multiple expected utility constraints. By introducing a corresponding Lagrange function, a saddle-point theorem of the utility constrained optimization is derived. The existence of a constrained optimal policy is characterized by optimal action sets specified w...

l AVERAGE COST SEMI - MARKOV DECISION PROCESSES

^ The Semi-Markov Decision model is considered under the criterion of long-run average cost. A new criterion, which for any policy considers the limit of the expected cost Incurred during the first n transitions divided by the expected length of the first n transitions, is considered. Conditions guaranteeing that an optimal stationary (nonrandomized) policy exist are then presented. It is also ...